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Abstract 

The purpose of this research study was to investigate if and how digital technologies could be 
used to support a triad-approach for student assessment in higher education. This triad-approach 
consisted of self-reflection, peer feedback, and instructor assessment practices in a pre-service 
teacher education course at a Canadian university. Through online surveys, journal postings, and 
post-course interviews the study participants indicated that digital technologies could be used to 
effectively support such a triad-approach only if students were more actively involved in the 
assessment process and the course instructor placed a greater emphasis on formative assessment 
practices. 

Resume 

Le but de cette recherche etait d’etudier dans quelle mesure les technologies numeriques 
pouvaient etre utilisees pour soutenir une approche triadique de revaluation des etudiants 
universitaires. Cette approche tripartite consistait dans Tautoreflexion, la retroaction par les pairs 
et les pratiques d’evaluation de Tenseignant dans un cours de formation des futurs enseignants au 
sein d’une universite canadienne. A Taide de sondages en ligne, d’annonces de journaux et 
d’entrevues post-formation, les participants a T etude ont indique que les technologies 
numeriques pouvaient etre utilisees pour soutenir efficacement une telle approche triadique, a 
condition que les etudiants soient impliques plus activement dans le processus d’evaluation et 
que Tenseignant mette davantage Taccent sur les pratiques devaluation formative. 

Introduction 

A number of educational researchers have stated that assessment drives approaches to learning in 
higher education (Biggs, 1998; Hedberg & Corrent-Agostinho, 1999; Herman & Linn, 2013: 
Marton & Saljo, 1984: Ramsden, 2003; Thistlethwaite, 2006). Entwistle (2000) indicates that the 
design of the assessment activity and the associated feedback can influence the type of learning 
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that takes place in a course or program. For example, standardized tests with minimal feedback 
can lead to memorization and a surface approach to learning while collaborative group projects 
can encourage dialogue, richer forms of feedback, and deeper modes of learning. The purpose of 
this research study was to investigate if and how digital technologies could be used to support a 
triad-approach to assessment in higher education. This triad approach consisted of self-reflection, 
peer feedback, and instructor assessment practices in a pre-service teacher education course at a 
Canadian university. 

There has been a shift in the way teachers and researchers think about student learning in higher 
education, over the last two decades. Instead of characterizing learning as an acquisition process 
based on teacher transmission, it is now more commonly conceptualized as a process of students 
actively constructing their own knowledge and skills (Barr & Tagg, 1995; DeCorte, 1996; Taber, 
2011). Students interact with subject concepts, transforming and discussing them with others, in 
order to internalize meaning and make connections with what they already know. Tenns like 
“learning-centered,” which have entered the vocabulary of higher education, are one reflection of 
this new way of thinking. Even though there is disagreement over the precise definition of a 
learning-centered approach, the core assumptions are “active engagement in learning and learner 
responsibility for the management of learning” (Lea, Stephenson & Troy, 2003, p. 323). 

Despite this shift in conceptions of teaching and learning, a parallel shift in relation to 
assessment and feedback has been slower to emerge. In higher education, the assessment process 
is still largely controlled by and seen as the responsibility of teachers; and feedback is still 
generally conceptualized as a transmission process, even though some educational researchers 
have challenged this viewpoint (Boud, 2000; Sadler, 1998; Wiggins, 2012; Yorke, 2003). 
Teachers “transmit” feedback messages to students about what is right and wrong in their 
academic course work, about its strengths and weaknesses, and students use this information to 
make subsequent improvements. 

There are a number of problems with this transmission view of assessment and feedback. Firstly, 
if the assessment process is exclusively in the hands of teachers, then it is difficult to see how 
students can become empowered and develop the self-regulation skills needed to prepare them 
for learning outside higher education institutions and throughout life (Boud, 2000). Secondly, 
there is an assumption that when teachers transmit feedback infonnation to students these 
messages are easily decoded and translated into action. Yet, there is strong evidence that 
feedback messages are often complex and difficult to decipher, and that students require 
opportunities to actively construct an understanding of them (e.g., through discussion) before 
they can be used to regulate perfonnance (Ferguson, 2011; Higgins, Hartley & Skelton, 2001; 
Ivanic, Clark & Rimmershaw, 2000). Thirdly, viewing feedback as a cognitive process involving 
only transfer of information ignores the way feedback interacts with motivation and beliefs. 
Research shows that feedback both regulates and is regulated by motivational beliefs. For 
example, external feedback has been shown to influence how students feel about themselves 
(positively or negatively), and what and how they learn (Dweck, 1999). Fourthly, as a result of 
this transmission view of assessment, the workload of teachers in higher education increases year 
by year as student numbers and class sizes become larger. One way of addressing this issue is to 
re-examine the nature of assessment feedback, and who provides it (e.g., self, peer, and 
instructor), in relation to its effectiveness in supporting learning processes. 
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Self-Assessment 

Alverno College (2006) defines self-assessment feedback as “the ability of students to observe, 
analyze, and judge their own performances on the basis of criteria and to determine how they can 
improve it” (p. 1). This assessment process is often referred to as metacognition “thinking about 
one’s own thinking” (Costa, 1985). Akyol and Garrison (2011) have recently demonstrated how 
this notion of self-regulated learning or metacognition is “a collaborative process where internal 
and external conditions are being constantly assessed” (p. 184). In addition, they have described 
three dimensions of metacognition, which involve the knowledge, monitoring, and regulation of 
cognition. The knowledge of cognition refers to awareness of self as a learner and includes 
entering knowledge and motivation associated with the inquiry process, academic discipline, and 
expectancies. The monitoring of the cognition dimension implies the awareness and willingness 
to reflect upon the learning process. And, the regulation of metacognition focuses on the action 
dimension of the learning experience. It involves the employment of strategies to achieve 
meaningful learning outcomes. 

Two major criticisms of self-assessment in higher education are that students do not possess the 
necessary skills and experience to properly assess themselves and thus, this fonn of assessment 
is unreliable and simply leads to grade inflation (Rust, 2002). Conversely, others suggest that 
self-assessment is a key process for helping students to reflect, understand, and take action and 
responsibility for their learning (Brown, 2004). 

Peer Assessment 

The Foundation Coalition (2002) indicates “peer assessment allows students to assess other 
students (their peers) in a course. Peer assessment can also provide data that might be used in 
assigning individual grades for team assignments” (p. 1). The French moralist and essayist 
Joubert (1842) is attributed with the quote “to teach is to learn twice” and in an effective course 
experience all participants are both learners and teachers. The tenn “teaching” rather than 
“teacher” presence implies that everyone in the course is responsible for providing input on the 
design, facilitation and direction of the teaching process. 

A number of concerns have been raised about this assessment approach including students’ lack 
of confidence in the process, their ability to provide meaningful feedback, and pressure from 
peers to provide positive feedback and grades (Langan & Wheater, 2003). These issues are 
countered by those who emphasize that peer assessment provides students with richer and more 
authentic opportunities to leam from their peers (e.g., view and critique each other’s work) as 
well as potentially reducing teacher workload (Boud, 2007). 

Instructor Assessment 

Instructor assessment practices in higher education are often limited to high-stakes summative 
assessment activities such as mid-tenn and final examinations (Boud, 2000). The role of an 
instructor should be to provide ongoing and meaningful assessment feedback in order to help 
students develop the necessary metacognitive skills and strategies to take responsibility for their 
learning. 
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Thus, teachers have begun to place a greater emphasis on formative assessment practices (Gibbs, 
2006; Gibbs & Simpson 2004; Gorsky, Caspi & Trumper, 2006; William, 2011). Pask’s (1976) 
Conversation Theory of Learning suggests that learning takes place through our intrapersonal 
(inner voice) and interpersonal (external voice with others) conversations and that assessment 
feedback helps shape and regulate this dialogue in higher education courses. Nicol and 
Macfarlane-Dick (2006) have developed seven principles of good assessment feedback based on 
the work of Pask: 

Good feedback: 

1. Helps clarify what good perfonnance is (goals, criteria, standards) 

2. Facilitates the development of self-assessment and reflection in learning 

3. Delivers high quality information to students about their learning 

4. Encourages teacher and peer dialogue around learning 

5. Encourages positive motivational beliefs and self esteem 

6. Provides opportunities to close the gap between current and desired perfonnance 

7. Provides information to teachers that can be used to help shape teaching. 

The increased collaborative digital technologies such as blogs, wikis, and other social 
networking applications in higher education can provide an opportunity to reinforce these 
principles of good assessment feedback. The term Web 2.0 was coined by O’Reilly in 2005 to 
describe the trend in the use of World Wide Web technology to enhance creativity, infonnation 
sharing, and, most notably, collaboration among users. Brown and Adler (2008) add that the 
capabilities of these Web 2.0 tools have “shifted attention from access to information toward 
access to people” (p. 18). The focus of this research study was to investigate whether digital 
technologies could be used to support and enhance self-reflection, peer feedback, and instructor 
assessment practices in a pre-service teacher education course at a Canadian university. 

Methods of Investigation 

An action research (Stringer, 2007) and case-based method (Creswell, 2013) were utilized to 
investigate how digital technologies could support student assessment in higher education. There 
are various forms of action research and the framework defined by Gilmore, Krantz and Ramirez 
(1986) was utilized: 

Action research . . . aims to contribute both to the practical concerns of people in an 
immediate problematic situation and to further the goals of social science 
simultaneously. Thus, there is a dual commitment in action research to study a 
system and concurrently to collaborate with members of the system in changing it 
in what is together regarded as a desirable direction. Accomplishing this twin goal 
requires the active collaboration of researcher and client, and thus it stresses the 
importance of co-leaming as a primary aspect of the research process, (p. 161) 

There have been concerns about the validity of this methodology as it is often carried about by 
individuals who are interested parties in the research (i.e., instructors) and thus, potentially 
biased in the data gathering and analysis (Pine, 2009). The justification for action research 
counters this criticism by suggesting that it is impossible to access practice without involving the 
practitioner. Practice is action informed by values and aims, which are not fully accessible from 
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the outside. The practitioner may not even be wholly aware of the meaning of his or her values 
until she or he tries to embody them in her action (Kemmis, 2009). 

This approach consisted of quantitative (pre- and post-course online surveys) and qualitative 
(online journal entries and post-course student interviews) research methods to collect and 
analyze data from students enrolled in a pre-service teacher education course entitled, Current 
and Emerging Pedagogical Technologies. This is a second year course where students explore 
and investigate the potential for integrating digital technologies into their future teaching 
practice. 

Data Collection 

The principal researcher for this study was also the course instructor and thus, data was collected 
by an undergraduate student research assistant (USRA) in order to minimize perceptions of 
coercion and bias. The USRA invited all students enrolled in the course to be part of this 
research project and a total of 22 students participated in this study (96% response rate). The 
project received institutional ethics approval and the students signed an informed consent form. 
The consent form offered the participants confidentiality and the ability to withdraw from the 
study at any time. 

The data collection process began with a pre-course online survey that was designed by the 
principal research and has not been validated statistically (Appendix A). The purpose of this 
survey was to identify students’ initial perceptions about the value of self-re flection, peer 
feedback, and instructor assessment based on previous course experiences. The survey consisted 
of a mixture of Likert-scale and open-ended questions and the second version of the Free 
Assessment Survey Tool ( http://toofast.ca) was used to administer an online version of the 
survey. 

Throughout the semester the student participants were also asked to complete an online journal 
entry after each major assessment activity (total of 5 assignments). These journal entries asked 
students to explain how they made use of self-reflection, peer review, and instructor assessment 
feedback to improve each of the course assignments (n=22, 96% response rate). The Majarha 
ePortfolio system ( http://mahara.org) was used to facilitate this online journaling process. 

At the end of the academic semester, the students were asked to complete a post-course online 
survey about their perceptions of self-reflection, peer feedback, and teacher assessment based on 
their course experience (n=18, 78% response rate). This survey consisted of identical questions 
from the pre-course survey. 

Finally, the students were invited to participate in a 30-minute post-course interview with the 
USRA to discuss the course assessment practices as well as the preliminary survey and journal 
findings. Four students volunteered to be interviewed and these interviews were digitally 
recorded and transcribed by the USRA. 

Data Analysis 

A constant comparative approach was used to identify patterns, themes, and categories of 
analysis that “emerge out of the data rather than being imposed on them prior to data collection 
and analysis” (Patton, 1990, p. 390). The pre- and post-course online survey along with the 
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journal data were exported into MS Excel for descriptive statistical and thematic analysis by the 
USRA and the course instructor. The means were calculated for the pre- and post-study Likert 
scale questions and 2-tail /-tests were perfonned in order to determine whether or not the 
differences in the mean scores were statistically significant (p<0.05). The survey data was 
correlated with the students’ journal responses throughout the semester. At the end of the 
semester, a preliminary report was compiled and emailed to each of the student participants who 
were then invited to participate in a post-course interview to discuss the initial study findings. 
These interviews were digitally recorded and transcribed. 

Findings 

The study findings are reported based on student perceptions of the use of digital technologies to 
support self, peer, and instructor assessment activities to improve academic course work. 

Self-assessment 

The pre-course survey results indicated that students had a wide range of perceptions regarding 
the value of self-assessment feedback. One student indicated “I don’t find it too important to me. 
I see by my grades how I am doing instead of evaluating myself’ (Survey Participant 11) while 
another stated “I would rather get feedback from a teacher or a peer” (Survey Participant 6). A 
number of students commented that they did not have much previous experience with self- 
assessment activities and thus, “I can sometimes have a hard time recognizing where I can 
improve when I’m self evaluating” (Survey Participant 17). 

During the course, students used digital technologies to support several self-assessment activities 
such as Audacity (http://audacity.sourccforge.net/) for self-assessment narrations of project 
artifacts (e.g., digital stories created in MS Photostory ( http://microsoft-photo- 
storv.en.softonic.com/) , Google Blogger (http://www.blogger.com/home) for online journaling, 
and Google Sites ( https://sites.google.com/) for the creation of an ePortfolio. The students who 
participated in this study were asked to rate the value of self-assessment feedback before and 
after the course. The results are displayed in Table 1. 


Table 1: Students’ perceptions of the value of self-assessment feedback 



High/ 
very high 

Medium 

Low/ 
very low 

Mean 

Diff. 

p value 

Before course 

41% 

45% 

14% 

2.95 

+0.46 

0.2345 

(not significant) 

After course 

59% 

35% 

6% 

3.41 


These results suggest that some students had a higher perception regarding the value of self- 
assessment at the end of the course but approximately one-third of the students were still 
ambivalent about the use of this type of assessment feedback and the difference in the pre- and 
post-means was not statistically significant. These findings were confirmed in the post-course 
interviews when students were asked to describe how they used self-assessment feedback to 
improve their course work. One student stated that assessment was the responsibility of the 
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course instructor and “Personally I didn’t feel I needed to do it... I don’t really value my 
opinion on assignments once I’ve finished them” (Interview Participant 3). Conversely, another 
student described how self-assessment activities helped her internalize her learning “I could see 
how I did things, what worked and what didn’t. I could also see my goals and if I really got to 
where I wanted to be” (Interview Participant 1). This comment was echoed by Interview 
Participant 4 who indicated: 

When I started to analyze and critique my own work I started seeing areas for 
improvement. I always want to give myself a good self-evaluation so I made 
changes or modified certain parts of the assignment to feel comfortable about 
giving myself a fair but good evaluation. 

Peer Assessment 

The student participants expressed a number of concerns about peer assessment activities based 
on their previous course experiences in the pre-course survey. These issues ranged from 
frustration, confusion, and academic loafing to intimidation with the process. For example, one 
student stated “It was frustrating because it didn’t really mean anything. The teacher re-marked 
the assignment anyways . . .” (Survey Participant 15). Another student was confused by the peer 
assessment process as “I didn’t know if their feedback would be right or wrong” (Survey 
Participant 21). Several students commented about academic loafing “I feel sometimes my 
classmates may not be paying attention and just give marks based on the hope that people will 
grade them lightly” (Survey Participant 5) and how the fear of intimidation limits the quality and 
honesty of the peer assessment “Students are always intimidated when evaluating their peers in 
fear of giving them a bad mark” (Survey Participant 11). 

Digital technologies were used in the course to support a variety of peer assessment activities. 
These activities included using Google Docs ( www.docs.google.com/) for peer review of student 
lesson plans. The group tools in the Blackboard Learning Management System 
( http://www.blackboard.com/) to provide peer review of project artifacts (e.g., digital stories 
created in MS Photostory - http://www.windowsphotostory.com/) . And, Wikispaces 
( http://www.wikispaces.com/) for the co-construction and peer editing of online discussion 
summaries and class notes. The students were also asked to rate the value of peer-assessment 
feedback before and after the course (Table 2). 


Table 2: Students’ perceptions of the value ofpeer assessment feedback 



High/ 
very high 

Medium 

Low/ 
very low 

Mean 

Diff. 

p value 

Before course 

19% 

62% 

19% 

3.23 

+0.59 

0.0367 

(significant) 

After course 

53% 

23.5% 

23.5% 

3.82 


Before the course, less than twenty percent of the students had a high perception of the value of 
peer assessment feedback. Whereas, after the course a very slight majority of students had a 
more positive perception of this form of assessment feedback and the difference in pre- and post¬ 
course mean scores were significant. These results were tempered by the post-course interview 
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results. One student indicated “I can see how it would be useful but I found that my peers either 
gave me wrong feedback, like telling me to do it one way when clearly the assignment said to do 
it another way, or just told me something I already knew and was working on” (Interview 
Participant 1). Another student described how she used peer-assessment feedback “as a guide¬ 
line for my work. It was nice to have someone review my work in the middle of the process 
because it let me know that I was on the right track” (Interview Participant 4). And Interview 
Participant 2 stated that “Any time we are able to have more eyes on something to add 
suggestions, it is worthwhile to take advantage of it.” 

Instructor Assessment 

In the pre-course survey, students indicated they received a range of assessment feedback from 
their instructors, which was usually summative in nature. One student stated that “It all depends 
on the teacher” (Survey Participant 3) while another complained that “No one tells me anything 
and since I don’t have the same teachers for each course, I don’t really kn ow how to improve my 
course work” (Survey Participant 21). Several students stressed “that it is important to be able to 
adapt to requirements that others set for you” (Survey Participant 14). 

Interactive technologies were used by the course instructor primarily to provide students with 
formative assessment feedback. For example, the instructor gave initial assessment comments 
and grades on all assignments in digital format (e.g., student lesson plans in Google Docs, digital 
stories in MS Photostory, WebQuests in QuestGarden - http://questgarden.com/) . Students then 
had the opportunity to revise their assignments based on this feedback and resubmit final 
versions to their ePortfolios (e.g., Google Sites ) for summative assessment. It appears that this 
emphasis on fonnative feedback impacted students’ perceptions regarding the value of instructor 
assessment (Table 3). 


Table 3: Students’ perceptions of the value of instructor assessment feedback 



High/ 
very high 

Medium 

Low/ 
very low 

Mean 

Diff. 

p value 

Before course 

64% 

36% 

0% 

3.86 

+0.64 

0.006 

(significant) 

After course 

94% 

6% 

0% 

4.50 


These survey results indicate that almost all of the students involved in this study had a much 
higher perception of instructor assessment, after the course and the difference in pre- and post¬ 
course mean scores were very significant. In the post-course interviews, the students explained 
how they used the instructor’s formative assessment comments to improve their course work: 
“The feedback allowed me to re-examine how I did something and then go back to review and 
make changes where necessary” (Interview Participant 2). For several students, this was the first 
time they had received formative feedback from an instructor and one student commented that 
“... having an instructor give you a first mark, and then being able to go back and revise was 
really helpful in improving my work” (Interview Participant 1). Another student emphasized 
how he “really liked the fonnative feedback from the instructor because it allows you to improve 
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on similar assignments you might have to do again in the same class or maybe another course - 
sort of like a building block approach to learning” (Interview Participant 4). 

Discussion and Recommendations 

The student participants in this research study were asked to provide recommendations for how 
digital technologies could be used to design meaningful self, peer, and instructor assessment 
activities in their online journal assignment postings and the post-study interviews. 

Self-Assessment 

With regards to self-assessment practices, the students provided specific recommendations for 
how digital technologies could be used for grading rubrics and online journaling in higher 
education courses. 

Rubrics 

The Teaching, Learning, and Technology (TLT) Group (2011) define a rubric as “an explicit set 
of criteria used for assessing a particular type of work or performance. A rubric usually also 
includes levels of potential achievement for each criterion, and sometimes also includes work or 
performance samples that typify each of those levels” (n.p.). The participants in this study 
indicated that rubrics can be useful for clarifying assignment and assessment expectations only 
when students are actively involved in their co-construction. In the post-study interviews, one 
participant stated that without student involvement rubrics “can become simple check-lists, a 
way to make sure that you’ve covered everything the teacher wants for the assignment rather 
than what you really wanted to do and learn” (Interview Participant 3). Unfortunately, this 
comment suggests that without student involvement rubrics have the potential to support a 
surface rather than a deep approach to learning. 

In terms of recommendations, the students suggested that several types of digital technologies 
could be used to support the co-construction of assessment rubrics. These included applications 
such as Rubistar ( http://rubistar.4teachers.org/index.php) , Teachnology ( http://www.teach- 
nology.com/web tools/rubrics/) , and Google Docs. The students in this study preferred using 
Google Docs based on the simplicity and their familiarity with this tool. An example of a co¬ 
constructed assessment rubric for a lesson plan assignment is illustrated in Figure 1. 

The study participants also recommended that students should have the opportunity to practice 
applying the co-constructed rubric to previous completed course work and that they should have 
the ability to add one unique grading component or criteria (e.g., creativity). 

In addition, digital technologies can be used to provide a variety of options for students to self- 
assess themselves. For example, students can use Audacity (http://audacity.sourceforge.net/) , the 
open-source audio tool to create self-assessment narrations of how they achieved the various 
learning outcomes outlined in the rubric. The use of self-assessment audio feedback can be of 
powerful way for students to internalize their learning (Ice, Curtis, Phillips & Wells, 2007). 
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Assessment rubric for a l|esson plan assignment: 


I 


Component 

Beginning 

Developing 

Accomplished 

Your Score 

& 

Comments 

Peer Score 

& 

Comments 

Instructor 

Score & 

Comments 

Peer Review 

0.25 points 
Incomplete peer 
review of another 
student s lesson 
plan 

0.5 points 

Basic comments regarding: 

What you learned from 
reviewing this lesson plan? 

What you liked about this 
lesson plan? 

What recommendations or 
advice you shared with your 
colleague to help improve this 
lesson plan? 

1.0 points 
Substantive and 
reflective comments 
about what you 
learned from 
reviewwg another 
student s lesson 
plan, what you liked 
about the plan, and 
suggestions for 
improving this 
document. 




Self-Reflection 

0.25 points 
Incomplete 
self-reflection of 
the lesson plan 
assignment 

0.5 points 

Completed self-reflection 
scoring of the lesson plan 
assignment with basic 
comments for each component 

1.0 points 
Completed 
self-reflection 
sconng of the 
lesson plan 
assignment with 
substantive and 

reflective comments 

about each 





Figure 1. Assessment rubric for a lesson plan assignment 
(http://tinyurl.com/lessonpIanrubric) 


Online Journals 

Students in professional programs such as Teacher Education and Nursing are often required to 
maintain either a course or program journal. Online blogging tools such as WordPress 
( http://wordpress.org/) and Google’s Blogger ( http://www.blogger.com/) are commonly being 
used to support this type of self-assessment activity. 

Students in this study indicated online journals can be useful for self-reflection but that too often 
they can become a “boring and repetitive activity when I am simply being asked to reply to a set 
of teacher directed questions. Usually, I just post what I thi nk the teacher wants to hear not what 
I’m really thinking” (Interview Participant 2). Again, without student involvement this type of 
self-assessment activity can reinforce a surface rather than a deep approach to learning. 

The study participants strongly recommended that students should have much greater control 
over their online journal postings. They suggested that there should be more opportunities for 
“freedom of expression rather than conforming to a teacher set structure” (Interview Participant 
1). The students involved in this study proposed that their online journal assignment should be 
focused on processed-orientated postings that led to a final product such as an end of the 
semester self-reflection paper. And, that this paper could be assessed using a co-constructed 
rubric in Google Docs. 

In the post-course interviews, the student participants stated they thought the use of digital 
technologies to support self-reflection was a useful type of formative assessment but that 
summative assessment practices should be the responsibility of the course instructor. This is 
consistent with the findings from a study conducted by Sitzman et ah, (2010), which found that 
self-assessment techniques contribute to student motivation and satisfaction but lack of 
confidence with summative grading. 
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Peer assessment 

The student participants suggested a variety of ways that peer assessment activities could be 
enhanced through the use of digital technologies in the post-study surveys and interviews. They 
indicated the biggest barrier to completing this type of peer activity, outside of class, was finding 
a common time and place to meet. The students suggested that digital technologies could 
potentially be used to overcome this challenge. For example, the group areas in learning 
management systems such as Blackboard could be used to communicate and share documents 
about the peer assessment process for individual and group projects. These group areas usually 
consist of asynchronous (e.g., email and discussion board) and synchronous (e.g., chat) 
communication tools along with a file exchange function. 

The students also indicated that during the course they had been impressed with how easy it was 
to provide peer review feedback on written assignment by sharing Google Docs (Figure 2). This 
application allowed them to control who had commenting and editing privileges for their 
documents. 
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Garbage Lesson Plan 
Grade 4 

Garbage Audit 
Science / Division 2 
Total Class Time Required: 
January 27. 2011 


USE^ 

DUCE 

CYCLE 


90 minutes 


Resources 

• Computers with Internet access for the use of google docs in order to create a spreadsheet and the world wide 
web for relevant student research 


• Link to: City of Calgary - Easy Guide to Recycling 
http://www.calparv.ca/docqallerv/buisws/blue cait/pdf/Mue cart materials poster en.odf 

• Projection screen and projector hooked-up with a computer so the whole class can watch a YouTube video and 
later see the results of the garbage audit 

• Handout for recording data from the garbage audit 

• Scales for weighing garbage (one per group - so 4 - 5) 

• A few extra garbage bins or bags to sort garbage 

• Digital Camera to record what we find 


A 


Great use of images in your 
lesson plan!! 


How many (e.g one P« f 
student or one per group of 
four students)? 


Be sure to also include the 
URL for the video in the 
Resource section 


Jon. I could not find this 
handout? ■ 


Figure 2. Peer review of a writing assignment in Google Docs 

In addition, online journal applications such as Blogger could be used to provide peer review 
feedback on individual project work and wiki tools such as Wikispaces 
( http://www.wikispaces.com/ ). The history files of a wiki summary clearly demonstrate the 
contribution and critique that was made by each member of the group. 

A number of students involved in this study were also taking an introductory Biology course. 
They commented in their journals and post-course interviews on how the instructor in this course 
was using the University of California at Los Angeles’ Calibrated Peer Review 
( http://cpr.molsci.ucla.edu/) tool to teach them how to provide constructive feedback to their 
peers on the laboratory manuscript assignments for the course. The Biology instructor also used 
personal response systems (e.g., clickers) for study group quizzes and discussion prompts. 
Crouch and Mazur (2001) describe how clickers can be used to support a form of peer 
instruction. The process begins with the teacher posing a question or problem. The students 
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initially work individually toward a solution and “vote” on what they believe is the correct 
answer by selecting the desired numbered or lettered response on their clicker. The results are 
then projected for the entire class to view. For a good question, there is usually a broad range of 
responses. Students are then required to compare and discuss their solutions with the person next 
to them in the classroom in order to come to a consensus. Another “vote” is taken but this time 
only one response or clicker per group can be utilized. In most circumstances, the range of 
responses decreases and usually centers around the correct answer. As an alternative to this 
process, this Biology instructor also had groups of students generate the quiz questions in 
advance of the classroom session. 

While the student participants appreciated the ability of digital technologies to provide increased 
flexibility and communication opportunities to complete peer assessment activities, outside of 
the classroom, they had several concerns. First, a number of students expressed concern about 
their lack of experience with peer assessment in the post-study interviews. They strongly 
recommended that instructors should “provide guidance and a class orientation on how to give 
each other meaningful feedback” (Instructor 4). Another student suggested that there should be 
“opportunities for oral and written feedback” (Instructor 2). He thought that digital technologies 
were being used primarily to provide written peer feedback and that students should also be 
learning how to provide oral feedback to each other. This comment was echoed by a student who 
suggested that instructors should “provide class time to begin and conclude peer assessment 
activities” (Instructor 1). She believed that this combination of face-to-face and online 
interaction would help to build trust and accountability for the peer assessment process. 

The students in the post-course interviews also recommended that this form of assessment should 
be used in combination with self and instructor assessment practices in order to triangulate the 
feedback. 

Instructor Assessment 

In terms of instructor assessment, the student participants provided several suggestions about 
how digital technologies could be used to support these practices. The first idea was to have 
instructors use collaborative writing tools such as Google Docs to provide formative assessment 
feedback at check-points or milestones for individual or group projects. This would allow 
students to receive instructor feedback throughout the process of constructing the project rather 
than just focusing on summative assessment feedback for the final product. 

The students also encouraged instructors to take a portfolio approach to assessment. This would 
involve students receiving a second chance or opportunity for summative assessment on their 
course assignments. For example, students would initially submit and receive instructor 
assessment for each of the required course assignments. Throughout the semester, students 
would have the opportunity to revise these assignments, based on the initial instructor feedback, 
and then post them to their course or program portfolios for final summative assessment by the 
instructor. There are a range of e-Portfolio tools that can support this process ranging from the 
LiveText (https://www.livetext.com/) commercial application to the free Google Sites 
( http://sites.google.com/) tool. 

In addition, digital technologies can be used to support external expert assessment opportunities. 
For example, students can publically share critiques of academic articles by using blogging tools 
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such as WordPress and Blogger. The authors’ of these articles can then be invited to post 
comments about these critiques to the students’ blogs. External experts can also provide 
assessment feedback on individual or group presentations through the use of web-based video 
technologies. These types of presentations can be video recorded and either streamed live (e.g., 
Livestream - http://www.livestream.com/) or posted to a video sharing site such as YouTube 
( http://www.youtube.com/) . The external experts can then provide assessment feedback in either 
synchronous (e.g., real-time audio) or asynchronous formats (e.g., online discussion forums) to 
the students. Figure 3 illustrates a video recording of an individual student presentation that has 
been posted to YouTube. 



Figure 3. Example of a video recording of student presentation posted to YouTube 

Besides providing ideas on how digital technologies could be used to support instructor 
assessment activities, the student participants in the interview sessions also had three 
recommendations for faculty members. The first recommendation was that instructors should 
“focus on providing students with ongoing formative assessment feedback rather than on just 
summative midterm and final examination comments” (Interview Participant 2). The second was 
that instructors should strive to “provide oral feedback in addition to their written assessment 
feedback. For example, instructors could request that students meet them during office hour 
sessions to orally debrief about assignments” (Interview Participant 4). Finally, Interview 
Participant 3 emphasized “Fet us provide instructors with more feedback on their assignments 
and teaching practice throughout the semester, not just at the end” and he recommended Angelo 
and Cross’ (1993) book on Classroom assessment techniques: A handbook for college teachers 
to facilitate this process. This focus on assessment being a process that involves two-way 
communication is consistent with Pask’s (1976) Conversation Theory of Teaming. 
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Conclusion 

Throughout this study, the student participants were asked to comment about how they used self, 
peer, and instructor assessment feedback to improve their course work. One student commented 
that “I used the self-reflection for checking my work and making sure I had everything I needed. 
I used peer-review for a different perspective on my work and I used instructor feedback to 
understand how I could improve my work” (Interview Participant 4). Another student stated that 
“Self reflection showed me what I liked about my work and what needed to be improved, peer 
feedback gave comments on what could be done better and then instructor feedback gave ideas 
on how the assignment could be fixed up to get a better mark” (Interview Participant 2). 

In addition, there were numerous comments in the student online journals about how digital 
technologies helped them integrate these three forms of assessment into a triad approach (Figure 
4). 


Self-Reflection 



Peer Feedback Instructor/External Expert 

Figure 4. Using digital technologies to support a triad approach to assessment 

For example, students were using rubrics, blogs, and online quizzes to provide themselves with 
self-reflection and feedback on their course assignments. Students received further feedback on 
their course work from their peers through the use of digital technologies such wikis and 
clickers. Finally, instructors and in some cases external experts reviewed students’ ePortfolios 
and used video technologies to observe student performance, diagnose student misconceptions, 
and provide additional formative assessment feedback. 

An international call for a greater focus on assessment for learning, rather than on assessment for 
just measurement and accountability of student performance is well documented in the 
educational research literature (Yeh, 2009). The use of digital technologies to support an 
increased focus on formative assessment practices may lead to Hattie’s (2009) vision of a visible 
teaching and learning framework where “teachers SEE learning through the eyes of their 
students and students SEE themselves as their own teachers” (p.238). 
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Study Limitations 

The two major limitations of this study were the small sample size and the focus on self-reported 
data. The small sample size (n=22) meant that significance is limited for the analysis of the 
quantitative survey data and thus, the results cannot be readily generalized or transferred to other 
higher education course contexts. The surveys, journal entries, and interviews conducted in this 
study all relied on self-reported data, which was limited by the fact that it was only verified by 
the author of this study. This data may contain several potential sources of bias such as selective 
memory of the student participant (i.e., remembering or not remembering experiences or events 
that occurred at some point in the course) and exaggeration (i.e., the act of representing outcomes 
or embellishing events as more significant than is actually suggested from other data) (Brutus, 
2013). 

Further Research 

The findings from this study strongly suggest that students require direction and practice in order 
to properly conduct self and peer assessments using digital technologies. Likewise, instructors 
also need guidance and support in order to properly facilitate all three forms of assessment 
through the use of technology. Thus, further research needs to be conducted in order to 
determine what type of training interventions are effective in helping students and instructors use 
digital technologies to support a triad-approach to assessment. 
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Appendix A: Pre-Course Student Online Survey Questions 

Important Note: The purpose of this survey is to gather student responses that will help inform 
the types of assessment practices used in the Mount Royal University Education Program. 
Participation in this survey is voluntary and your responses will be kept confidential. Non¬ 
participation in this study will not jeopardize student progress in this EDUC2325: Understanding 
Current and Emerging Pedagogical Technologies course or the Education Program. Completion 
of the questionnaire below will constitute informed consent in this How do students make use of 
self-assessment, peer assessment and teacher assessment feedback to improve their academic 
course work? study. This study has been approved by the Mount Royal Human Research Ethics 
Board (HREB). 


Name:_ 

A: Self Assessment Feedback 

Alverno College (2001) defines self assessment feedback as the ability of students to observe, 
analyze, and judge their own performance on the basis of criteria and detennine how they can 
improve it. 

1. What kind of previous experience do you have with self-assessment activities (i.e. 
journals, learning logs, portfolios)? 

2. How did you make use of this self-assessment feedback to improve your academic course 
work (e.g., strategies, processes)? 

3. Based on your previous experience, how would you rate the value of this self-assessment 
feedback? 

1 (very low), 2 (low), 3 (medium), 4 (high), 5 (very high) 

4. Why? 

B: Peer Assessment Feedback 

Peer review is a process used for checking the work performed by one's equals (peers) to ensure 
it meets specific criteria. 

1. What kind of previous experience do you have with peer assessment activities (i.e. group 
work, presentations, papers)? 

2. How did you make use of this peer assessment feedback to improve your academic 
course work (e.g., strategies, processes)? 

3. Based on your previous experience, how would you rate the value of this peer assessment 
feedback? 

1 (very low), 2 (low), 3 (medium), 4 (high), 5 (very high) 

4. Why? 
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C: Teacher Assessment Feedback 

Teacher assessment feedback on student work. 

1. What kind of previous experience do you have with teacher assessment activities and 
feedback (i.e. mid-terms, papers, final exams)? 

2. How did you make use of this teacher assessment feedback to improve your academic 
course work (e.g., strategies, processes)? 

3. Based on your previous experience, how would you rate the value of this teacher 
feedback? 

1 (very low), 2 (low), 3 (medium), 4 (high), 5 (very high) 

4. Why? 

D: Integration of Self-Assessment, Peer Assessment and Instructor Assessment Feedback 

1. Do you have previous experience integrating all three forms of assessment feedback to 
improve your academic course work? (Yes/No) 

2. If yes, please explain how you have integrated all three forms of assessment to improve 
your academic course work (e.g., strategies, processes)? 


How Digital Technologies Can Support a Triad-Approach 


21 



CJLT/RCAT Vol. 39(3) 


Author 

Norman Vaughan. An educator and researcher with interests in blended learning, faculty 
development and K to 12 schooling, Dr. Norm Vaughan is a Professor at Mount Royal 
University in Calgary, Alberta. He is the co-author of the books Teaching in Blended Learning 
Environments: Creating and Sustaining Communities of Inquiry (2013) and Blended Learning in 
Higher Education (2008). Email: nvaughan@mtroyal.ca . 



This work is licensed under a Creative Commons Attribution 3.0 License. 


How Digital Technologies Can Support a Triad-Approach 


22 









